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Executive Summary 



B etween 2002 and 2007, the Chicago High School Redesign Initiative 
(CHSRI) opened 23 small high schools. Implemented in partnership 
with Chicago Public Schools (CPS), this initiative aimed to provide popu- 
lations of low-performing students in under-served areas of the city with 
high-quality, small high schools. These schools were formed (1) by converting 
large high schools into a number of small ones, which were called redesigned 
schools, or (2) by creating new-start schools. This initiative was supported by 
the Bill and Melinda Gates Foundation, with additional funding provided 
by local foundations. CHSRI ceased to exist as a separate entity in August 
2008, leaving 17 small high schools still in full operation in their original 
locations. 

Chicago was one of many cities across the country that adopted small 
school reform in order to address high dropout rates and low academic 
performance at the high school level. This nation-wide reform wave at- 
tracted much attention and fanfare. However, as early results across multiple 
settings indicated a lack of consistent improvement in test scores, interest in 
small school reform faded in many cities. Attention shifted to other models 
for school improvement and other possible solutions to address the poor 
performance of high schools. Yet, continuing research across the country — 
and in Chicago — now indicates that graduation rates are often better in 
small high schools. This finding has important implications not only for 
small school reformers and researchers but also for those working on other 
reform models, including turnarounds that are designed to fundamentally 
reshape very low-performing schools. 

CHSRI posited that small, more intimate schools would improve student 
outcomes through two main mechanisms. First, if students felt personally 
and academically supported and teachers had more opportunities to know 
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their students, students would attend school more regu- 
larly and would learn more of the material. Students 
would, therefore, have higher academic performance. 
Second, if teachers had a working environment with 
more opportunities for meaningful collaboration that 
was conducive to innovation, they would develop strong 
professional communities that would spur instructional 
activities. This, in turn, would lead to stronger instruc- 
tion and improved student performance. 

Our earlier work on this initiative indicated that 
most of the elements of the theory of action were 
present in the CHSRI schools but that they were not 
strong enough to bring about changes in instruction 
or improve student test scores. Our previous work, 
based on surveys of students and teachers, revealed 
that CHSRI students consistently reported better re- 
lationships with adults in their schools and indicated 
that they felt academically challenged and supported. 
Teachers consistently reported more positive working 
environments that were marked by collegiality, in- 
novation, and trust. However, these surveys did not 
indicate a consistent pattern of instructional strength. 
Furthermore, CHSRI teachers reported in interviews 
that they were teaching no differently in CHSRI 
schools than they had been before. 

Instead of elaborating on theory or mechanisms for 
change, this report focuses on student outcomes at 
CHSRI schools and how they compare to other similar 
CPS schools. We paid particular attention to gradua- 
tion rates and were guided by the following three sets 
of questions: 

1. Did the population of students served by CHSRI schools 

change over time as CHSRI created new schools? 

We find that CHSRI created and supported schools 
serving at-risk students, especially through the re- 
designed schools. The new-start CHSRI schools are 
more similar to CPS non-selective schools in general 
than they are to the redesigned schools, but neither 
set of CHSRI schools serves CPS students who enter 
high school with privileged backgrounds or strong 
academic records. 



2. On average, how did CHSRI schools compare to other schools 
serving similar students in terms of absences, academic 
achievement, and graduation? Have these differences 
changed overtime? 

We found that students in CHSRI schools performed 
as well as or better than similar students in other 
similar CPS schools on a number of important out- 
comes. CHSRI freshmen were engaged enough to 
have better attendance than their peers (although, on 
average, they still missed almost a month of school 
per year) and to earn better grades in core subjects 
than similar students (although their average is still 
slightly below a C) . They have typically been more 
likely to be on-track to graduate than similar fresh- 
men, although this difference is very small in the 
most recent year. And cohorts of CHSRI students 
have been more likely to persist to graduation; in the 
2004-05 cohort, 57.2 percent of CHSRI students 
graduated compared to 49 percent for similar stu- 
dents who attended similar non-CHSRI schools. 1 

However, students at CHSRI schools scored 
about the same on the ACT (and the Prairie State 
Achievement Exam) as academically and demo- 
graphically similar students at similar schools. And 
their average composite ACT scores remained low for 
the six years covered by this report; their average of 
15.4 or below was well below the ACT college readi- 
ness benchmarks, which range from 18 in English 
to 22 in math and 24 in science. 

3. To what extent did CHSRI graduation rates vary across 
schools? Were some CHSRI schools more effective at 
graduating the students they serve? And, were CHSRI schools 

more effective at graduating some students than others? 

When looking at graduation rates across CHSRI 
schools, we found some variation. However, the 
overall positive difference in graduation rates across 
CHSRI schools is not limited to a few schools; 
all but two CHSRI schools had graduation rates 
similar to or higher than what we would have pre- 
dicted, given the characteristics of the students they 
were serving. 



2 



Chicago High School Redesign Initiative 



Finally, CHSRI schools were very effective at 
graduating the students who were most at risk of 
not graduating. This is true for all of the cohorts 
for which we have data — particularly the earlier 
ones. Furthermore, by the third cohort, students at 
CHSRI schools were more likely to graduate than 
similar students at other schools, regardless of their 
risk factors when they started high school. 

Our findings show that this initiative did accom- 
plish much, but not all, of what it was intended to do. 
CHSRI schools seem to have created an environment 
that encourages student attendance and persistence 
in school. CHSRI students’ academic outcomes are 



similar or slightly better than those of similar stu- 
dents. However, being “slightly better” than similar 
students does not mean that these students are college 
ready. Many other school districts are facing the same 
problem: how to bring under-performing students to 
college readiness in the span of four years. Countless 
researchers and practitioners are searching for a repli- 
cable, scalable method to accomplish this formidable 
task. The CHSRI schools have gotten at least part of 
the equation: their students persist in school and they 
graduate. This foundation should be recognized and 
built upon — and not forgotten — as schools continue 
to find ways to accelerate academic achievement for 
their students. 
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Introduction 




T he Chicago High School Redesign Initiative (CHSRI) was an initia- 
tive that sought to improve educational opportunities and academic 
outcomes among under-served students in Chicago by creating high-quality, 
small high schools. These were not to be schools-within-schools: each had 
its own unit number, administrative staff, and budget. This initiative was 
supported by the Bill and Melinda Gates Foundation with additional fund- 
ing provided by local foundations, and was implemented in partnership 
with the Chicago Public Schools (CPS). Between 2002 and 2007, CHSRI 
opened and supported 23 such small high schools to serve populations of 
generally low-performing students in neighborhoods in need of high-quality 
schooling options. This initiative ceased to exist as a separate entity in August 
2008, leaving 17 small high schools still in full operation in their original 
locations. 

A research team at the Consortium on Chicago School Research (CCSR) 
at the University of Chicago has studied CHSRI since spring 2003, relying 
on both qualitative and quantitative methodologies. We published a number 
of reports and articles chronicling the successes and challenges faced by the 
schools as they grew to maturity. We explored implementation issues, school 
contexts, and a range of student outcomes. 2 

In this report we present a close look at the schools making up the 
initiative, showing the characteristics of the students served by CHSRI 
schools and examining the degree to which this initiative remained true 
to its original goal of serving the needs of under-served students as it grew 
and changed over time. We then update our analyses of student outcomes, 
presenting all six years of data from the cohorts of students who started at 
CHSRI schools in fall 2002 through fall 2007. These outcomes include 
freshman absences, grade point averages (GPAs), and on-track rates; 
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results from standardized tests taken during students’ 
junior year; and cohort graduation rates. Each of 
these analyses provides an indication of how well, 
on average, CHSRI students performed compared to 
similar students at non-CHSRI schools serving similar 
students. 

In addition, this report focuses in-depth on gradu- 
ation rates, which is a particularly critical outcome. 
Here we analyze whether individual CHSRI schools 
were more effective at graduating their students 
than other schools serving similar students. And we 
analyze whether CHSRI schools were more effective 
at graduating some types of students than others, to 
see, for example, if CHSRI had a different impact on 
weaker students than it did on stronger students. 

Prior Research 

CHSRI’s theory of action proposed that student 
outcomes could be improved through two main 
mechanisms. First, if students were attracted to a 
school’s theme or philosophy and felt personally and 
academically supported, they would attend school more 
regularly and learn more of the material. Therefore, 
they would perform at a higher academic level. Second, 
if teachers had a working environment conducive to 
innovation and collaboration that was supported by 
CHSRI and CPS, they would develop strong profes- 
sional communities with a focus on instructional 
improvement. In turn, instruction and, ultimately, 
student performance would improve. Although this 
initiative did not promise an overall reduction in 
class size, it did theorize that small school size would 
enhance the opportunities for meaningful collabora- 
tion and conversations about teaching and learning 
among teachers, increase the likelihood that teachers 
would get to know their students better, and improve 
and strengthen both student-teacher and student- 
student relationships . 3 

Much of our prior work examined pieces of the 
theory of action through surveys, observations, and 
interviews. This prior work showed that CHSRI 
students consistently reported better relationships 
with adults in their schools and indicated they felt 
academically challenged and supported. They also 



reported stronger connections with other students 
and with the school, as the theory predicted, although 
students in the early cohorts did not seem to have 
been especially committed to the focus or theme of 
their new schools. Additionally, CHSRI teachers con- 
sistently reported having more extensive knowledge of 
their students and a more positive working environ- 
ment, marked by collegiality, innovation, and trust. 
However, teachers reported that they were teaching 
no differently in CHSRI schools than they had been 
before, and our analyses of student and teacher sur- 
vey reports did not find that classroom instruction 
was much different in CHSRI schools than in other 
schools serving similar students. 

Our prior work also focused on student outcomes, 
since improving student performance was the main 
purpose of this initiative. We analyzed early indica- 
tors, such as freshman on-track rates and freshman 
absences. We also looked at cumulative dropout rates 
and test scores for juniors. These earlier analyses 
showed that early cohorts of CHSRI freshmen were 
absent fewer days and were slightly more likely to 
be on-track than similar freshmen at schools serv- 
ing similar students. CHSRI schools had three-year 
cumulative dropout rates that were significantly lower 
than those of non-CHSRI schools serving similar stu- 
dents. As this initiative matured, our work indicated 
that graduation rates among the first cohort of students 
to attend CHSRI schools for four years were better than 
the graduation rates of similar students at non-CHSRI 
schools. However, this prior work also found that 
academic achievement, as measured by test scores was 
consistently low; it was not much different than that of 
similar students at similar non-CHSRI schools. 

These findings mirror what other researchers 
across the country have found: small schools seem 
to provide benefits related to the closer relationships 
among students and between students and adults . 4 
One especially salient finding is that students at these 
small schools graduate at higher rates. However, the 
hoped-for improvement in academic achievement, as 
measured by test scores, has not occurred. This is also 
similar to emerging findings about charter schools; 
in a study of charters in eight states, researchers from 
RAND Corporation found that test score achievement 



6 



Chicago High School Redesign Initiative 



gains at charter schools are similar to those of students 
at traditional public schools, although their graduation 
rates are higher . 5 

This lack of overall academic improvement was 
one factor that led the Gates Foundation to change 
its emphasis to include a specific focus on curriculum 
and instruction . 6 It has also been a factor that has led 
CPS to a renewed strong emphasis on instructional 
leadership and instructional change . 7 

Questions Guiding this Report 

In this report, we examine the schools and students 
making up CHSRI over time, update past analyses of 
student outcomes with additional years of data, and 
explore graduation rates in more detail than we have 
in other work. We address the following questions: 

Did the population of students served by CHSRI 
schools change over time as the initiative matured? 

On average, how did CHSRI schools compare to 
other schools serving similar students in terms of 
absences, academic achievement, and graduation? 
Did these differences change over time? 

To what extent did CHSRI graduation rates vary 
across schools? Were some CHSRI schools more 
effective at graduating the students they serve? And, 
were CHSRI schools more effective at graduating 
some students than others? 



Introduction 



7 




Characteristics of CHSRI Schools and 
the Students They Served 

C HSRI employed two different strategies for new school creation. Driven 
by a sense of urgency to get new schools started as soon as possible, 
CHSRI relied first on the strategy of creating redesigned schools. This strategy 
involved converting large, low-performing high schools that already had a 
history of creating informal learning communities based on student interest 
into a cluster of individual small schools that were housed within the same 
building. The second strategy relied on creating new-start schools. 

Of the 23 new, small CHSRI high schools, 12 were formed through 
redesign and 11 were formed through new-start schools. The blend of rede- 
signed schools and new-start schools changed over time. In the early years, 
CHSRI schools were mainly redesigned schools. By fall 2005, the proportion 
of redesigned schools in this initiative had fallen to 50 percent. When CHSRI 
ceased to exist as a separate entity in fall 2008, 17 schools remained open in 
their original locations and served students in all high school grades — eight 
redesigned schools and nine new-start schools. 8 

Table 1 details which schools were part of this initiative, when they opened, 
whether they were redesigned or new-starts, and what grades were included 
when they opened. Ten of the 1 1 new-start schools began with the ninth grade and 
added a new grade each year. The eleventh new-start school began with the seventh 
grade. Four of the 12 redesigned schools opened with ninth grade only. Two of 
the redesigned schools began with both ninth grade and an additional grade 
so they could build on already existing programmatic clusters, and six opened 
with grades nine through 12. 
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TABLE 1 



Schools making up the Chicago High School Redesign Initiative 



Redesign 


New-Start 




South Shore Campus 


Bowen Campus 


Orr Campus 




2002-2003 


• School of the Arts 3 

• School of 
Entrepreneurship 11 


• Chicago Discovery 
Academy 3 

• Bowen Environmental 
Studies (BEST) 


• Phoenix Academy 3 


2003-2004 


• School of Leadership 3 

• School of Technology 


• Global Visions Academy 


• Mose Vines Academy 3 


• Big Picture Back of the Yards 

• Big Picture Metro 


2004-2005 




• New Millennium School 
of Health 


• Applied Arts, Science 
and Technology 
(AAST) 3 

• EXCEL 3 


• Al Raby 

• Chicago Academy High School 




DuSable Campus 


Little Village/ 
North Lawndale 
Campus 


2005-2006 




• DuSable School 
of Leadership 

• Bronzeville Scholastic 
Academy 

• Daniel Hale Williams 
Prep School of 
Medicine 11 


• Social Justice 

• Infinity Math 
and Science 

• World Language 

• Multicultural Arts 



Note: No superscript indicates that the school opened with ninth grade only, 
a Opened with grades 9 and 11. b Opened with grades 9 and 10. 
c Opened with grades 9 through 12. d Opened with grade 7. 



There were, of course, many differences in early 
school formation between the two strategies. The 12 
redesigned schools were formed by converting three 
large comprehensive schools (Bowen, South Shore, 
and Orr) into 12 small schools in a phased-in process; 
however, the large schools did not immediately cease 
operation. Some incoming freshmen chose the small 
schools and others chose the comprehensive school 
during the early years of this initiative. These were 
neighborhood schools, meaning that all students who 
lived in the attendance area could attend the school 
while students who did not live in the attendance area 
could apply for any additional spaces that were avail- 
able. In this strategy, many of the teachers in the new 
schools had been teachers in the large “host” school be- 
fore it was converted. All of the large host schools were 
on probation when they were converted. One of them, 
Orr, had been “reconstituted” in the fall of 1997; all of 
them had gone through “intervention” starting in fall 



2000. 9 As would be expected, the redesigned schools 
that opened in these buildings had to confront a 
negative image and some staff morale challenges. 

The 1 1 new-start schools followed a different pat- 
tern. Seven were housed in shared space in two differ- 
ent buildings. Four were housed in the Little Village/ 
North Lawndale Campus, which was a new build- 
ing that had been designed specifically for these new 
schools. Three shared space with each other and, for 
the first year only, with graduating seniors from 
DuSable, a school that had been phased out. Two (Big 
Picture Metro and Chicago Academy) shared space 
with elementary schools. Two (A1 Raby and Big Picture 
Back of the Yards) had their own buildings. 

Because the new-start schools started from scratch, 
it was not possible for their faculty to come from any 
single location, as was the case in general with the 
faculty at the redesigned schools. Only the schools 
at Little Village, created at least in part due to neigh- 
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borhood demands, were considered neighborhood 
schools; the other seven had city-wide enrollment, 
meaning that incoming students had to apply to 
attend. None of the 11 new-start schools had to over- 
come a history of poor past performance. Although 
A1 Raby and the schools at DuSable were housed in 
schools that had been previously closed, a number of 
years had passed since these closed schools had last 
accepted freshmen; thus, these new-start schools were 
not seen as continuations of the schools in which they 
were housed. 

The literature on small schools indicates that it is 
more difficult and takes longer to create successful 
schools through redesign than through starting new 
ones, 10 at least in part due to some of the differences 
noted here — faculty hiring patterns, school reputa- 
tion from the past, and whether students needed to 
apply or could enroll “by default” based on where they 
lived. In Chicago, geography also played a role. The 
12 redesigned schools served three specific neighbor- 
hoods. The 11 new-start schools were located across 
the city in six different locations. 



CHSRI Students 

CHSRI schools were intended to provide educa- 
tional opportunities for students in under-served 
neighborhoods — neighborhoods marked by signifi- 
cant educational need. 11 Table 2 summarizes student 
characteristics for CHSRI schools as a whole and for 
other CPS non-selective schools at three points over 
the life of this initiative: 2002-03, 2004-05, and 
2007-08. Overall, students at CHSRI schools showed 
lower prior achievement, were more likely to be receiv- 
ing special education services, were more likely to be 
old-for-grade, and were more likely to have changed 
schools in the years immediately prior to high school. 
Because of their geographic locations, the earliest 
CHSRI schools served a population that had a high 
proportion of African American students. 

Table 2 also differentiates CHSRI redesigned schools 
from CHSRI new-start schools. Here we see that the 
population served by the redesigned schools was con- 
sistently more vulnerable than the student population 
of CPS overall or of the new-start schools. In fact, the 



TABLE 2 

Demographic characteristics of first-time freshmen 





2002-03 


2004-05 


2007-08 




CHSRI 




CHSRI 




CHSRI 






All 


All Other 


All 


Redesigned 


New- 


All Other 


All 


Redesigned* 


New- 


All Other 




(Redesigned) 


Nonselective 


n=16 


n=12 


start 


Nonselective 


n=20 


n=11 


Starts** 


Nonselective 




n = 5 


HS 






n=4 


HS 






n=9 


HS 






n = 63 








n = 62 








n=62 




Percentage Receiving Special Education Services 




24.2 


18.3 


25.5 


26.7 


20.0 


18.2 


17.6 


21.0 


13.7 


16.4 




Percentage Old for Grade 




25.1 


24.9 


34.9 


37.4 


23.4 


25.9 


28.8 


36.7 


19.4 


24.3 




Percentage with Two or More Moves in the Three Years Prior to High School 




19.3 


14.1 


20.1 


21.1 


15.4 


13.7 


10.9 


15.5 


5.6 


8.2 




Racial Composition Percentage 


African 

American 


82.6 


51.2 


85.6 


90.6 


63.1 


53.0 


76.6 


92.4 


57.9 


49.1 


Latino 


16.1 


37.7 


11.5 


8.6 


24.8 


36.8 


20.9 


7.2 


37.2 


41.4 


White 


1.1 


8.1 


2.2 


0.4 


10.7 


7.3 


1.8 


0.0 


3.8 


6.6 



' Phoenix, one of the initial redesigned schools, left this initiative in 2005; it is not included here. 

'* Both of the Big Picture schools started to phase out in 2006, when they stopped accepting freshmen; they are not included here. 
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new-start schools compared favorably with CPS over- 
all; especially by 2007, new-start schools served fewer 
students who were old-for-grade, were receiving special 
education services, or had changed schools in the years 
immediately preceding high school. 

Since we noted such demographic differences be- 
tween students in CHSRI and non-CHSRI schools, 
as well as between students in CHSRI redesigned and 
CHSRI new-start schools, we also wanted to focus 
more closely on achievement differences among stu- 
dents at the time they began high school. To measure 
the achievement of incoming first-time freshmen, we 
used composite scores on the EXPLORE test. Students 
in CPS take the EXPLORE test, which is one of a series 
of tests created by ACT, in the fall of their freshman 
year. Because it is given so early in the school year, it 
provides a good picture of the pre-high school achieve- 
ment level of incoming freshmen. 

As Figure 1A shows, the initial cohorts of schools 
served some of the lowest-performing students in 
Chicago. The median CHSRI school in academic years 
2002, 2003, and 2004 had students whose average 
composite ninth grade EXPLORE score was around 
12.3, more than one point below the median for similar 
non-CHSRI schools. Indeed, in those years, the average 
EXPLORE score of the CHSRI school with the very 
highest level of incoming achievement only slightly 
surpassed the median achievement level of all of the 
non-CHSRI schools. Furthermore, in those years of 

H I ■ 

How to Read Figures 1A and IB 

Figures 1A and IB depict the distribution of schools 
based on their average composite EXPLORE scores 
among first-time ninth-graders. In Figure 1A, the 
squares represent the median school achievement 
among non-CHSRI schools; this is the point at 
which half of the schools have higher average ninth 
grade EXPLORE scores, while half have lower av- 
erage ninth grade EXPLORE scores. Similarly, the 
circles represent the median achievement among 
CHSRI schools. In Figure IB, the diamonds rep- 
resent the median EXPLORE score for redesigned 



the lines with the circles tells us that there was little 
variability among CHSRI schools. Starting in academic 
year 2005, however, the median average composite 
EXPLORE scores for CHSRI schools went up and 
the distribution widened; this indicates that CHSRI 
schools were more different from each other than they 
had been earlier. By 2007, both the median score and 
the amount of variation across CHSRI schools started 
to look very similar to non-CHSRI schools. 

Because this increase in the incoming achievement 
level of CHSRI ninth-graders roughly coincided with 
the introduction of six new-start schools serving fresh- 
men and because of the demographic differences we 
noted in Table 2, we explored whether there was a dif- 
ference in the average prior achievement level between 
redesigned CHSRI schools and new-start CHSRI 
schools. As Figure IB shows, there had indeed been 
such a difference. In 2004, new-start schools (denoted 
by the lines with triangles) had higher levels of prior 
achievement than redesigned schools. 12 Because there 
were so few new-start schools that year, however, their 
relatively higher achievement scores were not enough 
to change the overall CHSRI average shown in Figure 
1A. Starting in 2005 when the six new-start schools 
serving freshmen opened, there was a notable difference 
in both ninth grade composite EXPLORE averages and 
in the distribution of those test scores. In that year and 
in the succeeding years, the median achievement level 
of CHSRI new-start schools was the same as, or higher 



CHSRI schools, while the median score for new-start 
CHSRI schools is represented by the triangles. In 
both Figure 1A and IB, the top of each line represents 
the school with the highest average level of incom- 
ing school achievement and the bottom of each line 
represents the school with the lowest average level of 
incoming school achievement. Therefore, the length 
of the lines describes how much variability there is 
among the schools in terms of their average incoming 
freshmen scores. 
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FIGURE 1 A 

On average, CHSRI schools serve many of the lowest-performing CPS students 
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FIGURE IB 

First-time ninth grade students in new-start CHSRI schools consistently have higher prior achievement than 
first-time ninth-graders in redesigned CHSRI schools 
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than, the achievement level of the highest redesigned 
CHSRI schools. Thus, the rise in mean incoming 
achievement in 2005-06 noted in Figure 1A is solely 
attributable to new-start schools in that year. Incoming 
achievement remained very low at redesigned schools 
over this whole time period. 

It is worth comparing Figures 1A and IB to note 
that by 2007 the nine new-start schools (including 
A1 Raby, Chicago Academy, and the seven schools 
in the Little Village/North Lawndale and DuSable 
campuses) had incoming achievement levels that were 
more similar to non-CHSRI schools than to redesigned 
CHSRI schools. 

As Figure IB and Table 2 indicate, the two sets 
of schools created by the two CHSRI strategies were 
serving quite different populations of students. While 
the redesigned schools served a very vulnerable popula- 
tion, some of the risk factors of students in the new-start 
schools looked more like non-CHSRI schools than 
their fellow redesigned CHSRI schools. In some ways, 
the redesigned schools were at a double disadvantage: 



they were started under a process that was widely seen 
as more difficult and they were serving a population 
of students with more challenging backgrounds. 

Given these demographic and prior achievement 
differences, it would not be surprising if students at 
CHSRI new-start schools had stronger outcomes than 
students at CHSRI redesigned schools. Table 3 illus- 
trates these differences by presenting some of the out- 
comes this initiative was trying to impact for students 
in redesigned schools and new-start schools. 

In conclusion, the students served by CHSRI schools 
did change over time as this initiative employed dif- 
ferent strategies for school creation. And, while the 
new-start schools seemed to have stronger outcomes, 
they also started out with stronger students. The next 
chapter examines (1) whether the performance differ- 
ences between redesigned schools and new-start schools 
remain after we consider their populations and (2) 
whether CHSRI schools overall perform better than 
other schools after we take the differences in their 
students into account. 



TABLE 3 

Students at CHSRI new-start schools have stronger outcomes than students at CHSRI redesigned schools 





2004-05 


2005-06 


2006-07 


2007-08 


Freshmen Absences-Days/Year 

• Redesigned Schools 

• New-Start Schools 


22.6 (1,195) 
7.4 (232) 


21.7 (1,368) 
6.0 (523) 


N/A 


N/A 


Freshmen Core GPA 

• Redesigned Schools 

• New-Start Schools 


1.58 (1,274) 
2.15 (239) 


1.55 (1,370) 
2.21 (714) 


1.60 (1,168) 
2.27 (708) 


1.57 (1,004) 
2.24 (769) 


On-Track to Graduate 

• Redesigned Schools 

• New-Start Schools 


52.9% (1,248) 
77.8% (234) 


52.3% (1,366) 
78.0% (710) 


52.6% (1,155) 
77.4% (707) 


51.8% (977) 
72.7% (763) 


Junior Year Test Scores-ACT 

• Redesigned Schools 

• New-Start Schools 


14.8 (490) 
N/A 


14.6 (721) 
15.4 (48) 


14.8 (737) 
16.2 (275) 


14.8 (875) 
16.2 (724) 



Note: Numbers in parentheses are the number of students in each group. 
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Average CHSRI Effect Over Time 

T able 3 showed that there were differences in outcomes between rede- 
signed schools and new-start schools. However, those differences largely 
disappeared after we included the differences in the incoming students served 
by the two groups of CHSRI schools as part of our analyses; we found that 
while students in the new-start schools did slightly better than similar stu- 
dents in the redesigned schools, these outcome differences were not large 
enough to be statistically significant. This means that if we compare students 
who were similar in their special education status, their age relative to their 
grade, their mobility prior to high school, their race /ethnicity, and their 
prior achievement, but who differed in whether they attended a new-start 
or a redesigned CHSRI school, there is no appreciable difference in their 
number of days absent, their likelihood of being on-track, their core GPA, 
or their ACT score. 13 

While there are no significant differences in outcomes among similar 
students in the two types of CHSRI schools, there are indeed significant 
differences among students who attend CHSRI schools and similar students 
who attend non-CHSRI schools serving a similar population. In this section, 
the graphs show CHSRI raw scores in blue. These scores represent an average 
across all CHSRI students — those at redesigned schools and those at new-start 
schools. The black line represents outcomes achieved by similar students at 
similar non-CHSRI schools. 



CONSORTIUM ON CHICAGO SCHOOL RESEARCH AT THE UNIVERSITY OF CHICAGO 



15 




Freshman Year Absences 



FIGURE 2 



In every year for which we have absence data (from 
2002 to 2006), first-time freshmen at CHSRI schools 
have been absent about 10 fewer days per year than 
students attending similar non-CHSRI schools (see 
Figure 2). In each year, this gap is statistically significant, 
meaning it is highly unlikely that the difference could 
have happened by chance. Although CHSRI ninth- 
graders are absent fewer days than students with similar 
backgrounds at other schools, they still miss between 
three and four weeks of school each year. 

Freshman Core Grade Point Averages 

Over time, core GPAs for first-time freshmen at 
CHSRI schools have been slightly higher than those 
earned by similar students at similar non-CHSRI 
schools, averaging between about 1 .7 and 1 .9, which is 
slightly less than a C average. In most years the differ- 
ence between CHSRI and similar non-CHSRI students, 
although very small, is large enough that it is statistically 
significant (see Figure 3). 

Freshman On-Track Rates 

The freshman on-track rate is a key performance 
indicator that shows whether students are on a path 
toward graduation. Students are “on-track” if they 
have earned five credits and have received no more 
than one semester F in a core course. Previous CCSR 
research has found that students who were on-track 
at the end of their freshman year were four times 
more likely to graduate in four years than off-track 
students. 14 Historically, CHSRI ninth-graders have had 
higher on-track rates than similar students at similar 
non-CHSRI schools, although the difference typically 
has not been statistically significant (see Figure 4). 
Furthermore, the difference has been shrinking over 
time. In 2007-08, the on-track rate of freshmen in 
CHSRI schools (61.0 percent) was almost the same 
as the rate for similar students at similar non-CHSRI 
schools (60.7 percent). 



Freshmen at CHSRI schools were absent fewer days than 
similar students at similar non-CHSRI schools 



Average Number of Days Absent for First-Time Freshmen 
CHSRI Students and Similar Students at Other Similar Schools 




2002-03 2003-04 2004-05 2005-06 2006-07 2007-08 2008-09 
■ Other CPS Schools } CHSRI Schools 
Note: ** p<0.01 ; * p<0.05; ~ p <0.10 



FIGURE 3 

Although the average core GPA in CHSRI schools was higher 
than in similar schools, it remained less than a C average 
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FIGURE 4 



Junior Year Test Scores 



CHSRI ninth-graders had better freshman on-track rates 
than comparable students until 2007, when the on-track 
rates converged 



On-Track to Graduate First-Time Freshmen 
CHSRI Students and Similar Students at Other Similar Schools 
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All CPS juniors take the ACT as part of the state’s No 
Child Left Behind high school test, the Prairie State 
Achievement Exam (PSAE). ACT scores have been 
empirically linked to college performance and can be 
used to predict the likelihood that students will suc- 
ceed in college-level courses. As such, ACT scores have 
become a crucial metric for students and schools alike. 

In every year since this initiative started in 2002, 
the average ACT score for students at CHSRI schools 
has been almost identical to the average ACT score for 
similar students at similar non-CHSRI schools, aver- 
aging between 14.5 and 15.4 (see Figure 5). Although 
there has been a slight increase, these scores remain 
low. For context, in 2008 the average ACT composite 
scores were 17.6 at CPS and 20.6 in Illinois. 15 ACT’s 
college readiness benchmarks range from 18 for English 
to 24 for science. 16 



Note: ** p<0.01 ; * p<0.05; ~ p <0.10 



Four- Year Graduation Rates 



FIGURE 5 

ACT composite scores among CHSRI juniors were no 
different than scores for similar students, and they 
remained well below college readiness benchmarks 
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We show graduation data for the first three cohorts 
of CHSRI schools: students who started as freshmen 
in 2002, 2003, and 2004. Cohort one consisted of 
five redesigned schools. Cohort two consisted of the 
original five redesigned schools, plus four additional 
redesigns and two new-start schools. 

By the time the 2004 cohort entered high school, 
there were freshmen at 12 redesigned schools and 
four new-start schools. Regardless of the number of 
schools in the cohort or the ratio of new-start schools 
and redesigned schools, the four-year graduation rate 
increased steadily across cohorts, and the difference 
between CHSRI schools and similar non-CHSRI 
schools was statistically significant at all three time 
points. Figure 6 presents the graduation rates for the 
first three cohorts. 



■ Other CPS Schools CHSRI Schools 



Note: ** p<0.01; * p<0.05; ~ p <0.10 
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FIGURE 6 

CHSRI students were more likely to graduate than 
similar students at similar non-CHSRI schools 

Four-Year Graduation Rate 

CHSRI Students and Similar Students at Other Similar Schools 




■ Other CPS Schools CHSRI Schools 



Note: ** pcO.OI; * p<0.05; ~ p <0.10 



Earlier reports on this initiative have noted this 
positive trend. Even when we only had three-year 
cumulative dropout rates for two cohorts and four- 
year graduation rates for one cohort, it was evident 
that students at CHSRI schools persisted in school 
at higher rates than similar non-CHSRI students. 17 
Now that we have more years of data, we see that the 
second cohort (nine redesigned schools and the two 
very small Big Picture new-start schools) continued 
to follow this same pattern, as did the third cohort 
(11 redesigned schools and four new-start schools). 
As Figure 6 shows, the positive difference for CHSRI 
schools, when compared to statistically similar students 
and schools, has ranged from 7 percentage points to 
more than 9 percentage points. However, as Figure 6 
also shows, these graduation rates remained below 60 
percent for all cohorts. 
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Focusing on Graduation Rates: 

Beyond “Average” 

I n Chapter 2, we concentrated on whether there was a “CHSRI effect” on 
a number of student outcomes. That analysis provided the average effect 
on each outcome across all CHSRI schools. 

In this section, we move beyond the average CHSRI effect and dig more 
deeply into high school graduation. Whether students persist to graduation 
is of vital importance both for the individual and for society as a whole. 
High school graduates earn more than $260,000 more than dropouts over 
their lifetimes; 18 in the aggregate, dropouts from the class of 2008 will cost 
the country more than $319 billion in lost wages. 19 Young people who have 
dropped out of school are far more likely to be incarcerated than young people 
who have finished high school. In fact, among youths who were 16 to 24 
years old in 2006-07, more than six times as many high school dropouts were 
institutionalized than high school graduates — a result with both individual 
and societal ramifications. 20 

Yet, the graduation rate in the United States for the class of 2002 has 
been estimated at only 71 percent; for students of color, it was 56 percent. 21 
Among a cohort of students in Chicago who were followed from age 13 until 
they were age 19 in 2002, 56.4 percent had graduated; only 50.3 percent 
of African Americans in the cohort finished high school. 22 Thus, it makes 
sense to probe more deeply into the details surrounding CHSRI’s positive 
graduation effect. 
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Differences in Graduation Rates 
Across Schools 

We began by exploring the degree to which individual 
CHSRI schools had stronger (or weaker) than pre- 
dicted graduation rates after taking into account the 
characteristics of the students they served. Schools in 
which graduation rates are more than approximately 
two standard deviations away from predicted values are, 
by our definition, doing either “better than expected” 
or “worse than expected.” 

Figure 7 provides the degree to which individual 
schools are doing better or worse at graduating their 
students than we would have predicted, given the stu- 
dents they serve. Figure 7 presents the whole distribu- 
tion of CPS schools for students who were freshmen 
in 2004, the last year for which we have graduation 
data. 23 Redesigned CHSRI schools are marked in 
blue; new-start CHSRI schools are marked in light 



blue. The length of the bars represents the degree of 
difference in graduation rates between each individual 
school and other schools that are serving similar stu- 
dents, measured in standard deviations. 

There were 15 CHSRI schools included in the 
2004-05 cohort: 11 were redesigned schools and four 
were new-start schools. As Figure 7 shows, most of 
them had graduation rates that were greater than or 
about equal to what we would have expected, based 
on the incoming characteristics of their students. 
Two schools had significantly lower graduation rates 
than expected. Both of these were redesigned schools, 
and both were converted into a turnaround school in 
fall 2008. Five schools had graduation rates that were 
significantly higher than we would have predicted, 
based on rates of other CPS schools. Two of these 
schools were redesigned; three of these schools were 
new-start schools (see Table 4 for a list of schools). 



FIGURE 7 

Most CHSRI schools have graduation rates that are higher than or equal to what would have been predicted 
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TABLE 4 



Schools with graduation rates that were greater than or less than expected, given the students they served 





Less Than Expected 


Greater Than Expected 


2002-03 Cohort 






2003-04 Cohort 


• Mose Vines Academy (Redesigned) 


• School of Technology (Redesigned) 


2004-05 Cohort 


• Mose Vines Academy (Redesigned) 

• EXCEL (Redesigned) 


• School of Technology (Redesigned) 

• Chicago Discovery Academy (Redesigned) 

• Al Raby (New-Start) 

• Chicago Academy (New-Start) 

• Big Picture-Williams (New-Start) 



TABLE 5 

Distribution of students by likelihood of graduating 



2004-05 Cohort 




CHSRI Schools 


Other CPS Schools 


Stratum 1 : The Least Likely to Graduate 


33 . 8 % 


18 . 4 % 


Stratum 2 


30 . 2 % 


19 . 7 % 


Stratum 3 


19 . 2 % 


20 . 5 % 


Stratum 4 


12 . 1 % 


20 . 5 % 


Stratum 5: The Most Likely to Graduate 


4 . 7 % 


21 . 0 % 



The story is similar in earlier cohorts: most CHSRI 
schools had graduation rates that were either greater 
than expected or about the same as expected. 

Therefore, we can say that there are differences 
among CHSRI schools in their ability to graduate their 
students. Furthermore, the overall positive difference in 
graduation rates across CHSRI schools is not limited to 
a few schools; by the third cohort, all but two schools 
had graduation rates that were at least as high as schools 
serving similar students. 

Differences in Graduation Rates 
Across Students 

In this section, we focus on the question of whether 
CHSRI schools were equally effective in graduating all 
types of students. We focused on the 2004 cohort to 
carry out this analysis. 24 We explored variation across 
students in the three steps described below. 

First, we grouped students according to background 
characteristics that are known to be related to gradua- 
tion. Looking across all CPS eighth-graders from spring 



2002 to spring 2004, we computed each student’s 
probability of graduating, based on background char- 
acteristics (race, gender, socioeconomic status, eighth 
grade achievement, age at the end of eighth grade, and 
changing elementary schools) immediately prior to 
entering high school. 

Second, we then divided all CPS students into five 
equal-sized groups, or strata, according to their graduation 
probability. Thus, each group was composed of students 
with a very similar likelihood of graduating, based on 
their background characteristics. Each group contained 
about 20 percent of the total number of students. 

Third, we then subdivided each stratum into 
CHSRI and non-CHSRI students. Table 5 shows the 
percent of CHSRI and non-CHSRI students in each 
category. Note that CHSRI students are proportion- 
ately over-represented in the lower two categories and 
proportionately under-represented in the upper two 
categories. This distribution is consistent with Figure 
1A and Table 2, which showed that CHSRI students 
entered high school with lower achievement and more 
risk factors. 
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Finally, we looked at the actual graduation rates 
of both CHSRI and non-CHSRI students in each of 
these strata. These actual graduation rates are plotted 
in Figure 8. As would be expected, the likelihood of 
graduating increases as the strata increase. 

Remember that students in each subgroup, whether 
CHSRI or non-CHSRI, had about the same likeli- 
hood of graduating, based on their pre-high school 
characteristics. 

What Figure 8 shows is that students in CHSRI 
schools were more likely to graduate than their peers 
in non-CHSRI schools, whether these students entered 
high school with high, medium, or low probabilities 
of graduating. Students with low probabilities of 
graduating, based on their background characteris- 
tics, benefited more than students who entered high 
school with characteristics that would indicate they 
were more likely to graduate. Among the lowest quin- 
tile (Stratum 1), the difference in graduation between 
CHSRI and non-CHSRI students was slightly more 
than six percentage points; the difference in the fifth -S 
quintile (Stratum 5) was about two percentage points. 52 
This indicates that although CHSRI schools were = 

° ' O 

effective in graduating students across all categories, Jj 
they were slightly more effective for students who 
entered with the lowest probability of graduating. 

Similar analyses for the two earlier cohorts of 
graduates showed that although there was an overall 
positive CHSRI effect, it was not equally distributed 



across students. Indeed, CHSRI schools were not very 
effective with some students (see Table 6) . 

Although going to a CHSRI school had an average 
positive effect on whether students graduated, our 
analyses showed that CHSRI schools were consistently 
more effective at graduating their most vulnerable 
students than other CPS schools. The relative advan- 
tage of attending a CHSRI school was more limited 
for students who entered high school with fewer risk 
factors. Indeed, in earlier cohorts, CHSRI schools 
were actually less effective than other CPS schools at 
graduating these students. 

FIGURE 8 

2004-05 cohort graduation rates by type of student 

CHSRI Students and Similar Students at Other Schools 
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TABLE 6 

Graduation rates by stratum, earlier cohorts 



2002-03 Cohort 2003-04 Cohort 





CHSRI 

Schools 


Other CPS 
Schools 


CHSRI 

Schools 


Other CPS 
Schools 


Stratum 1 : The Least Likely to Graduate 


33.0 


28.9 


36.1 


28.8 


Stratum 2 


42.2 


47.9 


56.8 


52.0 


Stratum 3 


59.1 


61.7 


67.3 


64.8 


Stratum 4 


78.7 


73.8 


76.4 


77.1 


Stratum 5: The Most Likely to Graduate 


82.8 


86.0 


78.8 


88.4 
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Conclusion 



A s this report shows, CHSRI did accomplish much — but not all — of what it 
J. was intended to do. First, it created and supported schools serving at-risk 
students, especially through the redesigned schools. The new-start CHSRI 
schools are more similar to CPS non-selective schools in general, but neither 
set of CHSRI schools serves CPS students who enter high school with strong 
academic skills or advantaged backgrounds. 

Second, students in CHSRI schools — whether redesigned or new- 
starts — performed as well as or better than similar students in similar non- 
CHSRI schools on a number of important outcomes. Whether they went to 
redesigned or new-start schools, CHSRI freshmen were engaged enough to 
have better attendance than their peers (although, on average, they still missed 
almost a month of school a year) and to earn better grades in core subjects 
than similar students (although their average was still slightly below a C). 
They have typically been more likely to be on-track to graduate than similar 
freshmen, although this difference is very small in the most recent year. And 
cohorts of CHSRI students have been more likely to persist to graduation, 
although their graduation rates have remained less than 60 percent. 

Nevertheless, even after six years, this initiative did not succeed in rais- 
ing student test scores. In earlier reports, CCSR researchers found that this 
initiative was successful at creating schools where students reported having 
better relationships with their teachers and where they felt more supported 
by adults in the building than students in similar non-CHSRI schools. And, 
again in earlier reports, we found CHSRI was successful at creating schools 
where teachers reported a context marked by more trust, collaboration, and 
shared leadership than teachers in other similar schools. However, all of 
these positive features did not spark the types of instructional improvement 
activities that make a visible difference in students’ test scores. 
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These findings may signal that some mechanisms 
for improving student performance impact particular 
outcomes, while other mechanisms impact others. The 
stronger relationships between students and teachers 
documented in our earlier work may improve absences, 
dropout rates, and even graduation rates, while the fact 
that we did not find substantial differences in instruction 
may help explain the lack of growth in test scores. 

This report moved beyond examining CHSRI 
schools and students and the average CHSRI effect 
on outcomes to focus on the crucial outcome of high 
school graduation. While there were differences in 
graduation rates across CHSRI schools, almost all 
CHSRI schools graduated at least as many students as 
statistically comparable schools. Future work could 
tease out the common elements across all of these 
disparate schools to see what most strongly impacts 
graduation — whether it is, for example, school size, 
student-teacher relationships, peer relationships, or 
instructional leadership styles. 

Furthermore, this report points out that the difference 
in graduation rates between CHSRI students and their 
peers was largest among students with the lowest prob- 
ability of graduating. In fact, if students from the two 
quintiles least likely to graduate had all gone to schools 
with the same graduation rates as the CHSRI schools 
attended by their peers, CPS would have produced 435 
additional graduates from among its most at-risk students 
in a single year. If students across all quintiles had at- 
tended schools with the same graduation rates as their 
CHSRI peers, 832 more would have graduated. 

CHSRI, along with similar initiatives in many other 
school districts, set out to create schools that would 



accelerate learning to the point where under-performing 
students would be college ready in the span of four years. 
Countless researchers and practitioners are searching 
for a replicable, scalable method to accomplish this 
formidable task. The CHSRI schools have gotten at 
least part of this equation: their students are more likely 
to persist in school and graduate. Indeed, those who 
entered school with the lowest probability of persisting 
did so at greater rates than similar students at similar 
non-CHSRI schools. 

This positive effect on graduation makes CHSRI 
highly relevant to the national reform discourse, even 
as small schools have taken a backseat to other reform 
models. U.S. President Barack Obama and Secretary 
of Education Arne Duncan have aggressively promoted 
a “turnaround” reform model, a massive infusion of 
federal money for states and districts to create or restart 
new schools or to overhaul the staff and organizational 
structure of their existing lowest-performing schools. 
They have called with special urgency for turnarounds 
at high schools with very low graduation rates. Yet, 
despite the attention the turnaround model has 
garnered, it remains largely untested. CHSRI was 
similarly an attempt to radically transform how low- 
performing schools operate, and its experience can 
provide insights for those planning and implementing 
turnarounds of high schools both in Chicago and 
nationally. CHSRI’s success graduating vulnerable 
students — while a partial success — should be embraced 
and built upon as funders, practitioners, policymakers, 
and the school reform community continue the search 
for powerful new ways to improve the nation’s lowest- 
performing high schools. 
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Appendix A: 

CHSRI Theory of Action 



Appendix A describes the theory of action behind 
CHSRI and presents a pictorial depiction of it (see 
Figure A). The framework explains the mechanisms 
through which the features, contexts, and players 
interact to promote the desired environments for 
teachers and students in these schools and how these 
elements promote curricular change and desired 
outcomes. This theory of action was developed by 
conducting interviews with key stakeholders (funders, 
district leaders, reform staff, teachers, and principals) 
and by examining written documents associated with 
the reform agenda (such as the request for proposal to 
create small schools and district statements regarding 
this initiative). 

The theory suggests that improved student outcomes 
result from numerous factors, both inside and outside 
the small schools. First, reformers assumed that if a 
district (Box 1) provided appropriate resources and 
policies, it could create small, voluntary schools, in 
which teachers and principals experience limited bu- 
reaucratic regulation (Box 2) . This, in turn, would help 
create a desirable teacher context for reform (Box 6) 
characterized by, for example, trust, collective respon- 
sibility, and teacher influence. This improved context 
for teachers would help spur instructional improvement 
activities (Box 7) . 



At the same time, resources and supports from both 
the CHSRI staff and the district staff (Boxes 4 and 5) 
would encourage instructional improvement activities 
(Box 7) by balancing their need to provide support, 
direction, and accountability systems with their need 
to protect the small schools’ autonomy and flexibility — 
since autonomy and flexibility are core components 
of the rationale for small schools. Reformers hoped 
that instructional improvement activities would lead 
to meaningful changes in instructional practice (Box 
8), and this would lead to improvements in student 
outcomes (Box 10). 

Reformers also believed that by choosing to be part 
of a school and sharing an interest in a common cur- 
ricular theme or instructional focus (Box 3), students’ 
sense of support and belonging would be enhanced and 
a supportive student environment (Box 9) would be 
created. These more personal relationships, knowledge 
of students’ needs and interests, and shared sense of 
community, combined with the ability to keep track 
of all students, were expected to help teachers provide 
greater academic and personal support, while hold- 
ing all students to higher expectations (Box 9). Even 
though this is important by itself, it was expected that 
being combined with high-quality instruction would 
make desired student outcomes more likely (Box 10). 
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FIGURE A 

CHSRI Theory of Action 
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Appendix B: 

Statistical Models and Variables Used 

H I Sir •! B 



Estimating the CHSRI Effect 

Freshmen Outcomes and Graduation 

To examine whether student outcomes at CHSRI 
schools are different from student outcomes at similar 
non-CHSRI schools, we use a two-level hierarchical 
model that we adjust for individual students’ char- 
acteristics and school-level characteristics. Level one 
represents students and level two represents schools. 

At level one, we control for previous performance as 
measured by the EXPLORE test that students take at 
the beginning of ninth grade, the social status in the 
census block where the student lives, the concentration 
of poverty in the student’s block, the student’s gender, 
the student’s race, whether the student has special 
education status, and whether the student is young 
or old for grade at the beginning of high school. We 
also include variables related to students’ mobility in 
elementary school and whether they attended a CPS 



school immediately prior to becoming ninth-graders. 
These slopes do not vary randomly at the school level. 
At level one, there are also six dummy variables indi- 
cating the year the data represents. These six year vari- 
ables are going to be a function of school characteristics 
for that year, and they are allowed to vary randomly at 
level 2. The school characteristics include an indicator 
of whether or not the school was a CHSRI school, the 
average incoming achievement level of all current stu- 
dents, the achievement level squared, and the school’s 
average student socioeconomic status. 

These models have no intercept. Therefore, each of 
the six year variables represents the mean for that year 
for the variable analyzed, adjusted for students’ and 
schools’ characteristics. Since we center all the variables 
around CHSRI students and CHSRI schools, the 
means for each year are the means for schools similar 
to CHSRI schools with similar students. 



The models and the variables in each level are described as follows: 

T| = P, (Student Prior Achievement) .. + (3 V (Student Prior Achievement 2 ) . + P v (Social Status) .. + 

IJ IJ IJ Z.J IJ DJ IJ 

P 4/ (Concentration of Poverty)^ + (C (Male)^ + Pg. (White) ^ + P 7/ (Asian ) .. + 

P g . (Native American)^ + (3 g . (Latino)^ + P 1Q . (Special Education)^. + |f ^ (No Economic Data ) .. + 

|f (No Student Prior Achievement) ~ + p 13 - (Young Began HS) .. + |f^ (Months Old Began HS) ;/ + 
Pi 5 y (Slightly Old Began HS) /; + p ]6/ (Moved 1 Time During ES) .. + P 17? . (Moved 2 Times during ES) ^ + 
P 18; - (Moved 3 Times during ES)^. + P |9? (Year 02-03) + P 2()/ (Year 03-04)^. + p 2]/ (Year 04-05)^ + 

P 22 / ( Year °5-06) •• + P 23/ (Year 06-07) •• + P 24/ (Year 07-08) .. + r .. 
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p# = Y^o for P = 1 through 18 

(3 19 . = y 190 + Y 191 (CHSRI 02—03) - + Y 192 (Mean Student Prior Achievement 02-03) + 

Y 193 (Mean Student Prior Achievement 02-03 2 ) ■ + Y 19 4 (Mean Social Status 02-03)^ + « ig . 
P 2Q . = Y 200 + Y)()i (C FI SRI 03-04) . + Y 202 (h4ean Student Prior Achievement 03-04). + 

Y 203 (Mean Student Prior Achievement 03-04 2 ) . + Y lo4 (Mean Social Status 03-04). + « 2Q . 
(3 21 = Y 2 io + Y?h(CHSRI 04 — 05)^ + Y-> 12 (Mean Student Prior Achievement 04-05) + 

Y 913 (Mean Student Prior Achievement 04 — 05 2 )y + Y^fMean Social Status 04-05)^ + u iy 
P 22 y = Y 220 + Y, 21 (CHSRI 05 — 06)^. + Y 2 i 2 (Mean Student Prior Achievement 05—06). + 

Y 223 (Mea n Student Prior Achievement 05 — 06 2 )^. + Y 224 (Mean Social Status 05-06). + « 22 . 
P 23 - = Y 230 + Y? 3 i(CHSRI 06-07). + Y 232 (Mean Student Prior Achievement 06— 07) • + 

Y 233 (Mean Student Prior Achievement 06-07 2 )y + Y 23 4 (Mean Social Status 06— 07) ■ + u iy 
P 24 = Y 2 4 o + Y 24 i(CHSRI 07-08) . + Y 242 (Mean Student Prior Achievement 07-08) . + 

Y 2 4 3 (Mean Student Prior Achievement 07— 08 2 ) . + Y 24 4 (Mean Social Status 07— 08) . + u.^ . 



For on-track to graduate indicator, 

cp.. 

Y. up.. " B(l, cp .) and T| .. = log — 

11 l ~ 11 v ' 11 7 'll & 

J J J J 1— cp 

'Vtj I 

for the number of days the student is absent during 
the year, 

Y- 1 A,.. " P( 1, A) and T) = log (A, ) 

and for the analysis of core GPA, V\.. = core GPA.. . 



The analysis on graduation rates follows the same 
model as described before, but the year dummy 
variables represent the entering cohort for that year. 
Graduation can only be analyzed for the first three 
cohorts; therefore, only three dummy variables (one 
for each cohort) are included in the model. For gradu- 
ation outcome, 

T.|cp.. -5(l,q>..) and r^. = log 



% 

1- <P;v 
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Eleventh Grade Test Scores 

To examine whether students’ test scores in eleventh students’ characteristics and school-level characteristics, 
grade at CHSRI schools are different from students’ As before, level one represents students and level two 
test scores at non-CHSRI schools, we use a two-level represents schools, 
hierarchical linear model adjusting for individual 

T) .. = B, (Student Prior Achievement) ■■ + B v (Student Prior Achievement 2 ) + (A . (Social Status) + 

IJ V IJ A/ l J 3J l J 

|) /(/ (Concentration of Poverty)^. + p-y (Male) ~ + P 6 y (White) ~ + [T (Asian) ~ + p^. (Native American)^ + 
(Latino)^. + P 10 y (Special Education)^. + (3 , y (No Economic Data ) + (3 ]2 ^. (No Prior Score) + 

P 13 - (Old) .. + P l4/ (Year 02-03) •• + P 15 - (Year 03-04) .. + P 16 - (Year 04-05) •• + P 17/ (Year 05-06) .. + 
P l8/ (Year 06-07) - + P 19; (Year 07-08) + r. 

P pj = y p0 for p = 1 through 13 

Pi 4 y = y l40 + y l4l (CHSRI 02-03)y + y l42 (Mean Prior Achievement 02-03)y + 

y l43 (Mean Prior Achievement 02-03 2 )y + y l44 (Mean Social Status 02-03)^ + u l4 - 
P 15 = y 150 + y 151 (CHSRI 03-04)^. + y 152 (Mean Prior Achievement 03-04)^. + 

y 153 (Mean Prior Achievement 03-04 2 )^. + y 154 (Mean Social Status 03-04)^. + u 1? . 

P 167 = T 160 + Yigi (CHSRI 04-05)^ + y 162 (Mean Prior Achievement 04-05)^ + 

y 163 (Mean Prior Achievement 04 — 05 2 )^ + y 164 (Mean Social Status 04-05)^ + « 16 - 
P 17 - = y 170 + y 171 (CHSRI 05-06) . + y 172 (Mean Prior Achievement 05—06) . + 

y 173 (Mean Prior Achievement 05-06 2 )^. + y 174 (Mean Social Status 05-06)^ + « 17 - 
Pi8y = Y 180 + Y 181 (CHSRI 06— 07) • + y 182 (Mean Prior Achievement 06—07)^ + 

y 183 (Mean Prior Achievement 06-07 2 ) + y lg4 (Mean Social Status 06-07)^ + « lg . 

P 19 . = y 190 +y 191 (CHSRI 07-08). + y 192 (Mean Prior Achievement 07-08)^ + 

Y 193 (Mean Prior Achievement 07-08 2 ) . + y 194 (Mean Social Status 07-08)^ + « ig . 

Where for the analysis of test scores, T 1 = ACT Score .. . 

1 'y l ) 
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Description of the Variables Used in the Models 



TABLE B1 

Student-Level Variable Used in Analyses 



Variable Name 


Description 


Where Used 


Student Prior Achievement 


Students’ elementary achievement was constructed based on each 
student’s composite EXPLORE score in 9th grade (for freshmen and 
graduations analyses) or reading scores on the Iowa Tests of Basic Skills 
in eighth grade (for eleventh grade test scores). 


All Analyses of 
Student Outcomes 


Male 


Gender was indicated by the variable MALE. 




Latino, White, 

Native American, Asian 


Race/ethnicity was indicated by a set of dummies. African American 
Students were the omitted category. 




Social Status, 
Concentration of Poverty 


Socioeconomic status included two variables. Both were based on data 
from the 2000 U.S. Census information on the census block group in which 
students lived. Students’ home addresses were used to link each student 
to a particular block group within the city, which could then be linked to 
census data on the economic conditions of the student’s neighborhood. 

Four indicators were used to construct these variables: (1) log of the 
percentage of families above the poverty line, (2) log of the percentage 
of men employed in the block group, (3) the average level of education 
among adults over age 21 , and (4) log of the percent of men in the block 
group employed as managers or executives. The first two of these were 
reversed coded and combined into the variable called Concentration of 
Poverty, while the other two indicators were combined into the variable 
called Social Status. The census data allow for a more accurate indicator 
of students’ economic status than a simple indicator of whether the student 
qualifies for free or reduced-price lunch. The vast majority of students in 
CPS qualify for free or reduced-price lunch, and there is wide variation in 
the economic status of students who qualify as low-income. Furthermore, 
by the time students reach high school age, proportionately more parents 
fail to apply for free and reduced-price lunch, and different schools treat 
this phenomenon differently. 




Special Education 


Whether a student was receiving special education services, based on 
administrative records. 




No Economic Data, 
No Test Data 


Missing values. If students were missing values on the socioeconomic 
variables or on their prior achievement, they were given values at the 
sample mean and given a dummy value of 1 on these two variables. 
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Variable Name 


Description 


Where Used 


Young Began HS, 
Months Old Began HS, 
Slightly Old Began HS 


Age on entry into high school was distinguished for first-time 
ninth-graders by these variables. “Young Began High School” is a 
dummy variable indicating whether a student was younger than age 14 
at the start of high school. “Months Old Began High School” gave the 
number of months older than 14 years, 8 months that a student was 
as of September 1 of the academic year. Students older than 14 years, 

8 months should have started school with the previous cohort if they 
were to follow school-system guidelines. “Slightly Old Began High School” 
is a dummy variable that indicates students between 14 years and 9 to 
11 months old when they began high school. 


Additional Variables: 
Analyses of First-time 
Freshmen and Graduation 


Move 1 Time During ES, 
Move 2 Times During ES, 
Move 3 Times During HS 


School mobility in elementary school was included in the models as a 
proxy of educational stability. Three dummy variables were constructed 
identifying students who moved once, twice, or three or more times in the 
three years prior to entering high school. More than 30 percent of students 
had moved once, 9 percent had moved twice, and 2 percent had moved 
three or more times during their immediately prior three years of schooling. 




Old 


Whether a student was old-for-grade was indicated by a dummy 
variable to indicate whether a student was older than what would be 
expected based on school system guidelines. 


Additional Variables: 
Eleventh Grade Test Scores 



TABLE B2 

School-Level Variable Used in Analyses 



Variable Name 


Description 


Where Used 


CHSRI 


Whether or not a school was a CHSRI school was indicated with the 
dummy variable CHSRI. 


All Analyses 


Mean Achievement 


School level achievement was computed by averaging the 9th grade 
composite EXPLORE scores (freshmen and graduation analyses) or 8th 
grade ITBS reading score (eleventh grade test score analysis) of all 
current members of a school’s student body, adjusting for cohort effects. 




Mean Social Status 


A description of how this variable was created at the individual level can 
be found in Table Cl . The school level indicator was an average of the 
social status of all students in the school based on the census block in 
which each lived. 
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Estimation of Differences in the Effects of 
Redesigned Schools and New-Start Schools 

To examine whether the effect of redesigned schools is 
different from the effect of new-start schools, we used 
the same model presented in the previous section where 
we add a dummy variable at the school level for each 
year when there are new-start schools. This variable 
takes a value of 1 if a CHSRI school was a new-start. 
The estimates on these dummy variables measure 
whether there are differences between the redesigned 
schools and new-start schools. 

Estimation of Individual School Effects 

To examine the individual school effects for gradua- 
tion instead of the aggregate CHSRI effect, we used 
the main model presented before with one exception: 
the CHSRI dummy variable is removed from the 
model. The school-level random variation (« ig . , u 1{] . , 



and «.. .) that can be estimated for each of the three 
cohorts in the graduation analysis after controlling 
for students’ characteristics and schools’ character- 
istics represents the individual school effects. These 
effects measured what schools contribute to students’ 
likelihood of graduating after taking into account the 
students that schools serve. These effects are plotted 
in Figure 7 after transforming them into standard 
deviation units. 

Measuring the CHSRI Effect by Type of Student 

To explore whether there was variation of the 
graduation CHSRI effect across different types of 
students, we estimated a propensity score model. This 
model estimates the probability of graduating for 
students, based on their achievement and background 
characteristics before joining high school. The model 
is as follows: 



Pr (Graduating in 4 Years = 1) . = p ! (8th Grade Achievement) . + P 2 (8th Grade Achievement 2 ) . + 

(3 , (Social Status) . + p 4 (Concentration of Poverty) i + p,(Male) . + 
(3 6 (White) ; . + |3 7 (Asian) ; + P g (Native American ) . + (3 g (Latino) z + 
P 10 (Special Education) . + (3 n (No Economic Data) . + 

P 12 (No 8th Grade Score ) i + P 13 (Young Began HS) ; . + 

P l4 (Months Old Began HS) ? . + P 15 (Slightly Old Began HS) . + 

P 16 (Moved 1 Time During ES) ; + P 17 (Moved 2 Times During ES) . + 
P lg (Moved 3 Plus Times During ES) ; . + p ig (Cohort 02—03) ; + 
P ?0 (Cohort 03-04) .+ p,j(Cohort 04-03) ; + r. 
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Each student gets an estimated propensity of graduat- 
ing in four years that is a function of their achievement 
and measurable characteristics. Students are grouped in 
five strata with 20 percent of the students in each, based 
on their propensity score. Table B3 shows the distribution 
of students into these five strata by cohort and by whether 

TABLE B3 

Distribution of students by propensity of graduating 



they attended CHSRI schools or other CPS schools. 

Finally the graduation rate was calculated for each of 
these groups of students and compared for CHSRI and 
non-CHSRI students within each stratum, since students 
within each strata have similar propensities of graduating 
in four years before the start of high school. 





2002-03 Cohort 


2003-04 Cohort 


2004-05 Cohort 




CHSRI 

Schools 


Other CPS 
Schools 


CHSRI 

Schools 


Other CPS 
Schools 


CHSRI 

Schools 


Other CPS 
Schools 


Stratum 1 : The Least Likely to Graduate 


28 . 5 % 


19 . 9 % 


37 . 5 % 


20 . 0 % 


33 . 8 % 


18 . 4 % 


Stratum 2 


27 . 4 % 


19 . 2 % 


26 . 0 % 


20 . 1 % 


30 . 2 % 


19 . 7 % 


Stratum 3 


23 . 7 % 


20 . 0 % 


18 . 8 % 


19 . 6 % 


19 . 2 % 


20 . 5 % 


Stratum 4 


12 . 6 % 


20 . 4 % 


14 . 0 % 


20 . 0 % 


12 . 1 % 


20 . 5 % 


Stratum 5: The Most Likely to Graduate 


7 . 8 % 


20 . 5 % 


3 . 8 % 


20 . 4 % 


4 . 7 % 


21 . 0 % 


Total Number of Students 


372 


20,723 


880 


20,276 


1,375 


21,475 



Note: CHSRI students are proportionately over-represented in the lower two categories 
and proportionately under-represented in the upper two categories. 
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Executive Summary 

1 . Students are on-track to graduate if they have successfully passed 
five courses and have no more than one semester failing grade in 
a core course. See Allensworth and Easton (2005, 2007). 

Introduction 

2. Kahne, Sporte, and Easton (2005); Kahne et al. (2006); Kahne 
et al. (2008); Kahne et al. (2009); Sporte, Correa, Kahne (2004); 
Sporte et al. (2003); Stevens (2008); Stevens with Kahne (2006); 
Stevens et al. (2008). 

3. See Appendix A for more detail and for a graphic representation 
of this theory of action. 

4. Mitchell et al. (2005); Rhodes et al. (2005); Shear et al. (2005). 

5. Zimmer et al. (2009). 

6. Greene and Symonds (2006); Robelen (2006). 

7. See, for example, Chicago Public Schools (2006); Reid (2005). 

Chapter 1 

8. Of the 23 schools created by CHSRI, six were no longer in 
existence in 2008. Of the four redesigned schools on one campus, 
one moved out into its own location in fall 2005; the other three 
were recombined and reconfigured as a large turnaround school in 
2008. Two of the new-start schools were phased out; due at least 
in part to economic challenges caused by their very small size, 
they closed when their first class graduated in 2009. 

9. See Hess (2003) for details about these earlier attempts at reform. 

10. Robelen (2006); Shear et al. (2005, p. 5). 

11. CPS, Office of Small Schools (2002). 

12. There were two new-start schools in 2003-04, but they did not 
have EXPLORE scores that year. Also one new-start school in 
2004-05 did not have EXPLORE scores. Phoenix Academy is 
not included in the analysis starting in 2005 when it moved to a 
new location and no longer considered itself part of CHSRI. 



Chapter 2 

13. See Appendix B for details about all models and methods used in 
this section. 

14. Allensworth and Easton (2005, 2007). 

15. Illinois State Board of Education (2008). 

16. When we look at overall PSAE scores in reading and math, the 
results are the same. Although there has been a slight increase 
in PSAE scores among CHSRI students, there is no difference 
between PSAE scores among students at CHSRI schools and 
their peers at similar non-CHSRI schools. 

17. Kahne et al. (2006). 

Chapter 3 

18. Rouse (2005). 

19. Alliance for Excellent Education (2008). 

20. Sum et al. (2009). 

21. Greene and Winters (2005). 

22. Allensworth (2005, pp. 12 and 21). Follow-up analyses as personal 
communication with authors. 

23. See Appendix B for an explanation on how these school effects are 
calculated. 

24. See Appendix B for a detailed description of the analysis. 
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